First-Order Algorithms for Convex Optimization with Nonseparate Objective and Coupled Constraints

In this paper we consider a block-structured convex optimization model, where in the objec-tive the block-variables are nonseparable and they are further linearly coupled in the constraint.For the 2-block case, we propose a number of first-order algorithms to solve this model. First,the alternating direction method of multipliers (ADMM) is extended, assuming that it is easyto optimize the augmented Lagrangian function with one block of variables at each time whilefixing the other block. We prove that O(1/t) iteration complexity bound holds under suitableconditions, where t is the number of iterations. If the subroutines of the ADMM cannot beimplemented, then we propose new alternative algorithms to be called alternating proximal gra-dient method of multipliers (APGMM), alternating gradient projection method of multipliers(AGPMM), and the hybrids thereof. Under suitable conditions, the O(1/t) iteration complexitybound is shown to hold for all the newly proposed algorithms. Finally, we extend the analysisfor the ADMM to the general multi-block case.